Papers with detection accuracy
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown promise in identifying deception, but their cognitive assistance potential remains underexplored. |
| Approach: | They propose a framework for LLM-based scam detection that bridges automated reasoning and human cognition. |
| Outcome: | The proposed framework outperforms GPT-4o in the Korean scam detection and phone scam simulations. |
Copied to clipboard
| Challenge: | a recent study shows that without artificially encouraging models to hallucinate, existing methods fall short . hallucinations are cases when the model generates output that is partially or fully unrelated to the source sentence. |
| Approach: | They propose a method that evaluates the percentage of the source contribution to a generated translation. |
| Outcome: | The proposed method improves detection accuracy for the most severe hallucinations by a factor of 2. |
Copied to clipboard
| Challenge: | Existing studies on toxic content in online communities are limited by the scarcity of data that align textual content with comprehensive social interactions. |
| Approach: | They propose a user-aware hate speech detection framework that effectively fuses textual semantics with social interaction representations to provide pragmatic context for disambiguation. |
| Outcome: | The proposed framework outperforms strong text-only baselines by over 3.6%, validating the critical role of social context in enhancing detection accuracy. |
Copied to clipboard
| Challenge: | Existing methods for hyperbole and metaphor detection focus on superficial text features, ignoring the associations of hyperbola and metaphor . Existing frameworks focus on identifying superficial text, focusing on superficial features . |
| Approach: | They propose an emotion-guided hyperbole and metaphor detection framework based on bidirectional dynamic interaction. |
| Outcome: | The proposed framework outperforms baseline methods on four datasets. |
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting can mitigate hallucinations by encouraging step-by-step reasoning, but its impact on halluciation detection remains underexplored. |
| Approach: | They conduct an empirical evaluation of CoT prompting in Large Language Models (LLMs) to examine their impact on hallucination detection methods. |
| Outcome: | The proposed method significantly affects the internal states and token probability distributions of the LLM. |
Copied to clipboard
| Challenge: | Prior work on out-of-domain detection requires in-domain task labels and is limited to supervised classification scenarios. |
| Approach: | They propose a method to construct out-of-domain detectors efficiently using pre-trained transformers. |
| Outcome: | The proposed method greatly improves out-of-domain detection ability in a more general scenario. |
Copied to clipboard
| Challenge: | Existing methods focus on distinguishing fully watermarked text from non-watermarked text, overlooking real-world scenarios where LLMs generate only brief segments within longer documents. |
| Approach: | They propose a method to detect watermarked segments in large documents using an anomaly extraction method and a local traversal. |
| Outcome: | The proposed method achieves a superior balance between detection accuracy and computational efficiency. |
Copied to clipboard
| Challenge: | Existing detectors use classifier-style probability signals or rely on rewriting, which can degrade quality and introduce new triggers. |
| Approach: | They propose to efficiently remove poisoned examples before or during fine-tuning . |
| Outcome: | The proposed method outperforms prior detectors on two machine translation datasets and one QA dataset. |
Copied to clipboard
| Challenge: | Existing methods for tamper detection rely on model stability, not inherently stochastic models. |
| Approach: | They propose a hypothesis-testing method for black-box tamper detection for LLMs . they propose regularized entropy-sensitive fingerprinting to enable efficient fingerprinting . |
| Outcome: | The proposed method achieves 98.80% detection accuracy under challenging conditions . it is based on a first-order surrogate for KL divergence to identify prompts most responsive to parameter perturbations. |
Copied to clipboard
| Challenge: | Hate speech detection datasets often use different annotation guidelines, resulting in inconsistencies . authors propose a topic-oriented approach to study generalization across popular hate speech datasets . |
| Approach: | They propose a topic-oriented approach to study generalization across popular hate speech datasets . they compare Transformer-based models in capturing topic-generic and topic-specific knowledge . |
| Outcome: | The proposed approach improves the reliability of hate speech detection on social media platforms. |
Copied to clipboard
| Challenge: | Existing methods for detecting fake news videos fall short due to lack of knowledge to verify the news is real or not. |
| Approach: | They propose a VLM-based framework for detecting fake news on short video platforms . they design four experts tailored to handle each scenario and integrate them into VLM . |
| Outcome: | The proposed framework outperforms current state-of-the-art models on two benchmark datasets. |
Copied to clipboard
| Challenge: | Text generated by Large Language Models (LLMs) now rivals human writing, raising concerns about its misuse. |
| Approach: | They propose a framework for sentence-level AI-generated text detection via style and context fusion. |
| Outcome: | The proposed framework outperforms baseline models in detection accuracy while exhibiting transferability and robustness. |
Copied to clipboard
| Challenge: | Existing audio deepfake detection datasets are outdated and lack generalization capabilities. |
| Approach: | They construct a new cross-domain audio deepfake detection dataset comprising over 300 hours of speech data that is generated by five advanced zero-shot TTS models. |
| Outcome: | The proposed models achieve 4.1% and 6.5% error rates in the cross-domain ADD dataset generated by five advanced zero-shot TTS models. |
Copied to clipboard
| Challenge: | prevailing taxonomies neglect robustness and honesty, yielding safer-on-paper but less useful systems. |
| Approach: | They propose a soft-gating pipeline where a guardian predicts a binary risk label plus a concise explanation and prepends this advice to the original query for re-inference. |
| Outcome: | The proposed model maintains safety while reducing over-refusal. |
Copied to clipboard
| Challenge: | Existing efforts to detect factually incorrect content are omitted by creators who subtly reshape impressions by omitting crucial background context. |
| Approach: | They propose a multi-stage pipeline that simulates preview-based and context-based understanding and a OMGuard pipeline that combines interpretation-aware fine-tuning and rationale-guided misleading content correction. |
| Outcome: | The proposed framework lifts an 8B model’s detection accuracy to the level of a 235B LVLM while delivering stronger end-to-end correction. |
Copied to clipboard
| Challenge: | Existing studies focus on evaluating large language models' ability to handle disagreement cases. |
| Approach: | They evaluate the performance of large language models in detecting offensive language at varying levels of agreement. |
| Outcome: | The proposed model improves detection accuracy and model alignment with human judgment by using disagreement samples in training. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have become central to NLP, demonstrating their ability to adapt to various tasks through prompting techniques. |
| Approach: | They probe the hidden layers of Large Language Models to identify where sentiment features are most represented and to assess how this affects sentiment analysis. |
| Outcome: | The proposed approach enables sentiment tasks to be performed with memory requirements reduced by an average of 57%. |
Copied to clipboard
| Challenge: | Existing methods for detecting text fragments are not suitable for contract discovery, since it requires manual definition of a few examples, followed by conventional information. |
| Approach: | They propose a task where legal clauses are extracted from documents, given a few examples of similar clauses from other legal acts. |
| Outcome: | The proposed task differs substantially from conventional NLI and shared tasks on legal information extraction. |
Copied to clipboard
| Challenge: | Multimodal large language models have demonstrated impressive capabilities in visual reasoning and text generation. |
| Approach: | They propose a multimodal large language model that captures deeper relationships between images and text . they propose CMIE, which uses a Coexistence Relationship Generation strategy and an AS mechanism to detect misinformation. |
| Outcome: | The proposed framework outperforms existing methods in detecting out-of-context misinformation. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been extensively utilized for health-related tasks, yet their performance in depression detection remains limited when relying solely on text input. |
| Approach: | They propose a system that leverages speech timing features for depression detection and reliable confidence estimation. |
| Outcome: | The proposed system outperforms text-based RAG systems in depression detection and confidence estimation. |
Copied to clipboard
| Challenge: | Existing methods for AD detection are too expensive and time-consuming to cover all potential patients. |
| Approach: | They propose a contrastive learning method to obtain effective text representations based on monolingual embeddings of BERT and a cross-lingual data augmentation method by building autoencoders to learn the text representation shared by both languages. |
| Outcome: | The proposed method outperforms other methods on a Mandarin AD corpus and achieves 81.6% detection accuracy. |
Copied to clipboard
| Challenge: | Recent advances in large language models have led to an increase in synthetic content generation . the ability to detect LLMs-generated content has become of paramount importance . |
| Approach: | They propose to provide a detailed overview of existing detection strategies and benchmarks, scrutinizing their differences and advocating for more adaptable and robust models to enhance detection accuracy. |
| Outcome: | The proposed model will be able to detect human-written content in real time. |
Copied to clipboard
| Challenge: | In response to adversarial text attacks, attack detection models have been proposed and shown to successfully identify text modified by adversaries. |
| Approach: | They propose to modify a text to cause the detection model to predict an attack while keeping the classifier correct. |
| Outcome: | The proposed model drops detection accuracy between 20 - 71 points while maintaining (or improving) classifier accuracy. |
Copied to clipboard
| Challenge: | Existing methods focus excessively on detection accuracy, neglecting the societal risks posed by high false positive rates (FPRs). |
| Approach: | They propose a Conformal Prediction framework that constrains the upper bound of false positive rates and introduces a real-time detection framework. |
| Outcome: | The proposed framework reduces false positive rates and improves detection performance. |
Copied to clipboard
| Challenge: | Existing methods for detecting hateful videos rely on opaque models with no insight into their decisions. |
| Approach: | They propose a lightweight, explainable video detection framework that distills "explainability" from LMMs into efficient Smaller Multimodal Models (SMMs) they use a self-grounded chain-of-thought mechanism to generate unbiased supervision signals for videos . |
| Outcome: | The proposed framework outperforms existing methods in detection accuracy and explainability on three video benchmarks. |
Copied to clipboard
| Challenge: | Existing methods for text watermarking ignore strong evidences embedded in low-entropy tokens, causing statistical measures to falsely indicate the absence of a watermark. |
| Approach: | They propose a Bayes' Rule derived watermark Detector which exploits watermark information from every token by leveraging the posterior probability of watermark’s presence. |
| Outcome: | The proposed method achieves 50% and 70% relative improvements over baselines in code generation and math problem-solving tasks. |
Copied to clipboard
| Challenge: | Recent studies show that human-written text is not distinguishable from synthetic text because of semantic errors or logical contradictions. |
| Approach: | They propose to analyze the forms of artifacts left by neural Text Generation Models by corrupting texts and replacing them with linguistic or statistical features. |
| Outcome: | The proposed method replaces text with linguistic or statistical features and improves the accuracy of the model. |
Copied to clipboard
| Challenge: | Recent advances in text-to-speech technology have enabled highly realistic voice generation, fueling deepfake attacks. |
| Approach: | They propose a framework for transcript-to-audio perturbation anti-spoofing that incorporates linguistic variation into detectors to investigate spoof detection. |
| Outcome: | The proposed framework can bypass commercial detectors by incorporating linguistic variation into the design of anti-spoofing systems. |
Copied to clipboard
| Challenge: | Existing evaluation approaches focus on mLLMs’ detection accuracy for binary classification tasks, which often fail to reflect the in-depth interpretive nuance of harmfulness across diverse contexts. |
| Approach: | They propose an agent-based arena-style evaluation framework that provides context-aware and unbiased assessment for mLLMs’ understanding of multimodal harmfulness. |
| Outcome: | The proposed framework reduces evaluation biases of judge agents and provides unbiased comparisons of mLLMs’ abilities to interpret multimodal harmfulness. |
Copied to clipboard
| Challenge: | Recent advances in Large Language Models have fostered a new class of generative linguistic steganography, claim “provably secure” by theoretically aligning the stego distribution with the language model’s natural distribution. |
| Approach: | They propose a framework that transforms the detection task from semantic classification to a statistical audit of the sampling mechanism. |
| Outcome: | The proposed framework breaks the security of AC and Meteor with high detection accuracy, whereas state-of-the-art semantic steganalyzers degrade to random guessing. |
Copied to clipboard
| Challenge: | Indirect Prompt Injection attacks can be exploited by LLMs that are embedded with external data. |
| Approach: | They propose a detection-based approach that leverages the behavioral states of LLMs to identify potential IPI attacks. |
| Outcome: | The proposed approach reduces the success rate of attacks to 0.03% on the BIPIA benchmark. |
Copied to clipboard
| Challenge: | Existing evidence suggests that LLMs are not able to detect scientifically unsound work from malicious or poorly designed research agents. |
| Approach: | They develop a framework that evaluates whether fabrication-oriented paper generation agents can deceive multi-model LLM review systems. |
| Outcome: | The proposed framework shows that fabricated papers achieve acceptance rates up to 18% . the framework shows only marginal improvements, with detection accuracy barely exceeding random chance. |
Copied to clipboard
| Challenge: | Existing defenses against forgery are inadequate for healthcare. |
| Approach: | They propose a large-scale benchmark for pre-hoc, evidence-grounded medical forgery detection using a doctor inspection guideline and gold edit locations. |
| Outcome: | Experiments show that the proposed solution can detect and explain medical scans with high fidelity and accuracy. |
Copied to clipboard
| Challenge: | Existing models have a performance gap of 20% between classifying fake news and real news, making them less suitable for practical deployment. |
| Approach: | They propose to adopt an LLM to generate fake news in three different styles, which are later incorporated into the training set to augment the representation of fake news. |
| Outcome: | The proposed model achieves state-of-the-art performance on two benchmark datasets and improves detection accuracy by 24.02% and 11.06% respectively. |
Copied to clipboard
| Challenge: | a new benchmark is constructed to evaluate the accuracy of large language models for tabular data . the benchmark uses direct, indirect, and Chain-of-Thought prompting . |
| Approach: | They propose a framework that uses prompting, self-verification and constraint-based rule execution to improve accuracy. |
| Outcome: | The proposed framework significantly improves accuracy and recall in tabular data. |
Copied to clipboard
| Challenge: | Existing sequential LLMs cannot be directly applied to DLMs, as their generation order is arbitrary. |
| Approach: | They propose a stability-aware constraint that allows watermarking only in stable contexts and a bit-controlled, unbiased modulation to preserve the original DLM output distribution. |
| Outcome: | The proposed scheme achieves stable watermarking with minimal quality impact while maintaining high detection accuracy and multi-bit capacity. |
Copied to clipboard
| Challenge: | Recent studies have shown that Large language models can detect factual inconsistencies in summaries but they lack the efficiency and explainability needed to be effective. |
| Approach: | They propose to decouple LLMs’ information extraction and reasoning capabilities to address key challenges and propose a framework for UIEFID to guide fine-tuned LLM methods in extracting unified structured information from documents and summaries. |
| Outcome: | The proposed framework improves the detection accuracy and reduces redundant reasoning on the AGGREFACT benchmark. |
Copied to clipboard
| Challenge: | Existing detectors that perform well on benchmark datasets have weaknesses that can be exploited to manipulate AI-text. |
| Approach: | They propose a framework that simulates high-temperature sampling effects through multiple normal-temperaturing generations, effectively evading detection. |
| Outcome: | The proposed framework reduces detector accuracy by an average of 82.5% while preserving high text quality. |
Copied to clipboard
| Challenge: | Existing video fake news detection benchmarks focus on the detection accuracy, while failing to provide fine-grained assessments for the entire detection process. |
| Approach: | They propose a process-oriented video fake news detection benchmark that evaluates MLLMs' perception, understanding, and reasoning capabilities in VFND. |
| Outcome: | The proposed model achieves sota performance on video fake news detection tasks. |
Copied to clipboard
| Challenge: | a new framework for hate speech detection addresses implicit hate speech by tailoring the detection process to dataset-specific attributes. |
| Approach: | They propose a framework to account for the dataset-specific characteristics of hate speech datasets. |
| Outcome: | The proposed framework improves detection accuracy and provides interpretable insights into the distinctive features of each dataset. |